Sina Weibo Incident Monitor and Chinese Disaster Microblogging

This paper describes the initial work on developing an all-hazards emergency event detector using messages obtained in near-real-time from the public timeline of the Chinese Sina Weibo microblogging service. The system filters target keywords corresponding to emergency events of earthquakes, floods, typhoons, fires and storms and then uses classifiers to identify messages from people experiencing the corresponding emergency event. Then, this study carried out experiments that compare the performance of four different classification methods and also explore to improve the classifier by the new training data captured by SWIM recently. After Chinese text pre-processing, feature selection and training set size, the experimental results demonstrate Random forests classifier could get best performance but need more long time to run in R, thus the potential to improve this classifier for setting up the SWIM system need to be explored in the future. While similar work has been reported using Twitter content, this is the first time these techniques have been applied to the Sina Weibo microblogging service for multiple emergency event types. This paper also outline the experience of accessing Sina Weibo messages, provide a summary of their structure and content, note the challenges faced in processing this text using Natural Language Processing packages and outline the developed website for users to view the processed messages. The long term aim is to develop a general emergency notification and monitoring system for various disaster event types in China reported by the public on Sina Weibo which can be used by the appropriate emergency services as a source of improved situational awareness. Subject Categories and Descriptors H.2.8 [Database Applications]: Data mining; K.4.1 [Public Policy Issues]: Privacy Hua Bai 1*, Xunguo Lin 2, Bella Robinsion 2, Robert Power 2 1School of Management, Harbin Institute of Technology Heilongjiang, 150001, China 2CSIRO Digital Productivity Flagship G.P.O. Box 664, Canberra, ACT 2601, Australia baihua1727@163.com General Terms: Text Mining , Chinese Text Classify